Twitter API Design Evaluation and Latency Budget

Introduction#

After familiarizing ourselves with Twitter's services and their endpoints, we'll focus on the last two aspects of designing the API in this lesson, namely how to meet non-functional requirements and how to estimate the response time of our Twitter API. Moreover, we’ll also discuss some interesting scenarios related to timelines and try to optimize the service using different approaches.

Non-functional requirements#

Let's discuss how we can achieve the non-functional requirements of the API for Twitter services.

Availability#

We ensure the availability of the system even during unexpected spikes (for instance, a celebrity’s Tweet needs to be delivered to millions of followers in a timely manner) by having loosely coupled services run separate tasks concurrently and statelessly. An example of such loosely coupled services is the usage of the pub-sub service between the Tweet service and the timeline service. The pub-sub service decouples our two main services and queues multiple concurrent Tweets during peak hours. Furthermore, we use a monitoring system that helps us detect anomalies, such as overloading the service due to excessive requests. To prevent excessive requests, rate limiting helps us reduce network traffic by restricting users' access to the Twitter API for a certain period of time. For example, the user can post a maximum of 15 Tweets per minute.

Reliability #

We use circuit breakers to identify and recover from bad situations as quickly as possible for our services. Also, we eliminate the single point of failure by routing the request to any available replica service. Furthermore, we use the backend for the frontend (BFF) approach for our API gateway to make it reliable and available because our services are used by different clients (mobile and website). For example, if the Twitter website is down (which is very rare), the service of the mobile application will not be affected by this downtime because the BFF handles each frontend or client type independently.

Scalability #

The stateless nature of the HTTP allows us to divide the load of incoming requests across multiple servers. Next, we use a mixture of push and pull models for the timeline updates, depending on user type (active or inactive), to serve a large number of users.

Security #

Twitter's service authenticates and authorizes users by using their credentials. The credentials must be encrypted through HTTPS/TLS. Typical users can authenticate themselves using the Authorization: Basic <encodedCreds> header. Moreover, we adopt the OAuth/OIDC code authentication with the PKCE mechanism for third-party logins (such as through Google, Apple, etc.).

Low latency#

For systems like Twitter, where the number of users and data increase daily, we maintain as low a latency as possible by pre-generating the timelines and storing them in the feed cache to serve the active users. Since the system is read-heavy and users are likely to read the most recent Tweets, it makes sense to keep them in the cache. Furthermore, we adopt cursor pagination, which] helps us to paginate the Tweets efficiently using the next-cursor pointer to avoid unnecessary load on the network, client, or server side. Also, links to popular data (such as images, videos, scripts, and so on) are generated dynamically, which directs the user to the nearest CDN. The client can download popular media content from the CDNs, which allows us to improve the response time. The following illustration shows how users can retrieve trends or Tweets related to them from the nearest CDNs instead of fetching them from the origin server.

Latency budget#

This section calculates the response time for our Tweet and timeline APIs. We will start by estimating the sizes of the request and response messages for each service. Next, we will head on to calculate the response times of each service.

Note: As discussed in the “Back-of-the-Envelope Latency Calculations” chapter, the average$RTT$ remains the same in the case of GET, regardless of the data size (due to the small request size), and the time to download the response varies by$0.4\ ms$ per KB. Similarly, for POST requests, the$RTT$ time changes with the data size by$1.15\ ms$ per KB after the base RTT time, which was 260 ms.

Tweet API#

The Tweet API stores a Tweet and returns the created Tweet attributes in response. We assume that our Tweet API allows us to post the following type of content in Tweets, as mentioned earlier:

1. Text, mentions, or hashtags—the length of which cannot be more than 280 characters

2. Up to 4 images

3. One video with a max size of 10 MBs

Content types 2 and 3 require first uploading the media files and then using only the mediaID in the request to post a Tweet. In order to efficiently utilize the network bandwidth, the returned response for these contains only the media links.

Request and response size#

Since the POST request will contain both media and Tweet text, we will estimate the sizes of each as follows:

• Media size and response time:  Since we have already estimated the response time of files in the file upload API, we will use the same procedure for the media file response time. Let us assume a user wants to post a Tweet that includes three images with some text. As we have seen above, we send a preflight request for uploading any media file prior to posting the Tweet along with the mediaID. Let's estimate the time taken to upload the media files. For this, assume each image file is 175 KB which takes around 414–560 ms  (estimated using the file upload API calculator) to upload. For the plausibility of our service, we will assume the maximum possible latency, which, in this case, is 560 ms.

• Tweet size: For posting a Tweet, the request headers take 1 KB and the request body takes 1.12 KBs because the request body comprises multiple data entities, such as tweetID, userID, content, mediaID, and other fields. The final request size is the summation of the request header and body, as shown below:

The response consists of the header and body sizes. We take the header size of 1 KB, and the response body size is 3 KB. Normally, the POST request takes a smaller response size than the request size. However, in our case, we're getting a response size slightly larger than the request size because we're getting the posted Tweet with other data entities, such as userID, tweetID, and information related to the mentioned profiles, hashtags, media, etc., in a response body.

Response time#

We calculated the message size for the Tweet request. Now, it's time to estimate the response time of our Tweet API. We take some values (say, the base time, processing time, and RTT) from the “Back-of-the-Envelope Calculations for Latency” chapter.

We can use the following calculator by entering the request and response sizes to get the Tweet API's minimum and maximum response.

A summary of the latency budget for a request for the Tweet service is shown in the illustration below:

Let's now move to the timeline service response time.

Timeline API#

The Tweet API returns only the recently posted Tweet in response. However, the timeline API returns the stream of Tweets with ads, people, and trends.

Request and response size #

We assumed that the request size for a GET request is 1.52 KB and the response size is 170 KB. The request consists of some essential headers and userId, at most. Let’s assume that the response body includes 80 Tweets, five trends, and two promoted ads. If the size of each Tweet is 2 KB and the size of the top five trends is 2.5 KB, whereas two ads and three user accounts are of sizes 5 KB and 1.5 KB, respectively, then the total size is equal to:

The size of the GET request is generally smaller than the response because the request body is empty. Therefore, we will only consider the response size to estimate the request's response time.

Response time#

Let's calculate the response time by setting the values of response size in the following calculator:

A summary of the latency budget for GET request for the timeline service is shown in the illustration below:

In this chapter, we designed services to post Tweets and view a timeline in the Twitter API. Additionally, we addressed the non-functional requirements of the service by employing the appropriate technologies. Finally, in the “Latency budget” section, we estimated that the service would respond in near real-time.